Building part compositions for hierarchical object recognition

The visual signature of objects in our world follow a particular blueprint: every object is composed of an arrangement of smaller visual "parts". These parts can be thought of on many different scales. An image of an car can be considered to be pattern of wheel, window, door, and various other compositional parts, or on a much more granular scale as a massive pattern of interconnected line segment parts. These levels of granularity lend themselves well to a hierarchical organization of visual blueprints: the car can be decomposed into doors and other similarly-scaled parts, each door can be decomposed into a window, a handle, and other features, and this reduction in scale can continue until a suitable base level of elemental parts is reached. In their 2007 paper, "Towards Scalable Representations of Object Categories: Learning a Hierarchy of Parts" [3], Fidler and Leonardis described an algorithm to implement such a categorization scheme with minimal human intervention. Given a large corpus of images, they showed their algorithm organizing subparts into patterns completely automatically from the bottom elemental layer to parts three layers higher. At this level, parts became too specialized to learn with a generic image corpus, so further learning was done on manually separated sets of images grouped by categories (e.g. faces, mugs, etc.). According to the authors, this was the only source of significant human intervention in the process. This work has attempted to recreate their algorithm from scratch. Unfortunately, this task was not successfully completed. Although the entire framework of the algorithm was written, automating the core pattern processing part of it proved to be a very nuanced and difficult task to complete. This paper will discuss the nature of this difficulty and describe the various successes and setbacks in developing a principled way to process the patterns in the data automatically.